A Hierarchical Classification of First-Order Recurrent Neural Networks
نویسندگان
چکیده
منابع مشابه
A Hierarchical Classification of First-Order Recurrent Neural Networks
We provide a decidable hierarchical classification of first-order recurrent neural networks made up of McCulloch and Pitts cells. This classification is achieved by proving an equivalence result between such neural networks and deterministic Büuchi automata, and then translating the Wadge classification theory from the abstract machine to the neural network context. The obtained hierarchy of ne...
متن کاملFirst-order versus second-order single-layer recurrent neural networks
We examine the representational capabilities of first-order and second-order single-layer recurrent neural networks (SLRNN's) with hard-limiting neurons. We show that a second-order SLRNN is strictly more powerful than a first-order SLRNN. However, if the first-order SLRNN is augmented with output layers of feedforward neurons, it can implement any finite-state recognizer, but only if state-spl...
متن کاملHierarchical Multiscale Recurrent Neural Networks
Learning both hierarchical and temporal representation has been among the longstanding challenges of recurrent neural networks. Multiscale recurrent neural networks have been considered as a promising approach to resolve this issue, yet there has been a lack of empirical evidence showing that this type of models can actually capture the temporal dependencies by discovering the latent hierarchic...
متن کاملHigher Order Recurrent Neural Networks
In this paper, we study novel neural network structures to better model long term dependency in sequential data. We propose to use more memory units to keep track of more preceding states in recurrent neural networks (RNNs), which are all recurrently fed to the hidden layers as feedback through different weighted paths. By extending the popular recurrent structure in RNNs, we provide the models...
متن کاملHierarchical self-programming in recurrent neural networks
We study self-programming in recurrent neural networks where both neurons (the ‘processors’) and synaptic interactions (‘the programme’) evolve in time simultaneously, according to specific coupled stochastic equations. The interactions are divided into a hierarchy of L groups with adiabatically separated and monotonically increasing time-scales, representing sub-routines of the system programm...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
ژورنال
عنوان ژورنال: The Chinese Journal of Physiology
سال: 2010
ISSN: 0304-4920
DOI: 10.4077/cjp.2010.amm037